Should we try to convert common types to a string? I think all values are strings in the first place, but I think we should avoid silently dropping a field in the event it is not a string.

Yep, can do. The options were to wait for #190 to be fixed (I've not looked at what is needed for this) at which point we'll be able to do this properly or convert everything to a string.

All these fields are not compliant with OTel naming guidelines. I would rather recommend to make this change in one leap to the attribute names that will be used going forward than just moving all the fields to attributes as is and changing them after that. This would require semantic conventions PR first.

Another concern I have is that I'm not sure we need all these fields. Many of them don't provide real value IMO. I think we can reduce number of attributes to essential ones. That's why I suggested to provide an option to not parse the journald payload in the issue, in case if user do want all of these fields as is .

cc @djaglowski

All these fields are not compliant with OTel naming guidelines. I would rather recommend to make this change in one leap to the attribute names that will be used going forward than just moving all the fields to attributes as is and changing them after that. This would require semantic conventions PR first.

Unfortunately, I think waiting for the semantic conventions blocks most of this PR for now, as the body will have to remain structured in order to hold most of the fields. I am on board with waiting though, as this is the best way to ensure we are only moving in the right direction.

Shall we:

1. Have a small PR now that just parses out severity
2. Make a PR to spec that proposes semantic conventions
3. Circle back to this after 2 is accepted

@gregoryfranklin, are you on board with this? Are you willing to propose the semantic conventions?

Another concern I have is that I'm not sure we need all these fields. Many of them don't provide real value IMO. I think we can reduce number of attributes to essential ones. That's why I suggested to provide an option to not parse the journald payload in the issue, in case if user do want all of these fields as is .

One design option that comes to mind is something like the preset option used for severity parsing. Basically, we can have some predefined sets of values that will be parsed. One of these sets is the default. The user can optionally choose another set and/or add additional fields to any set.

# just the default set
- type: journald_input

# default set plus a few extra fields
- type: journald_input
  fields:
  - USER_INVOCATION_ID
  - _SYSTEMD_USER_UNIT

# alternate predefined set plus a few extra fields
- type: journald_input
  field_set: some_other_set
  fields:
  - USER_INVOCATION_ID
  - _SYSTEMD_USER_UNIT

# only a few specific fields
- type: journald_input
  field_set: none
  fields:
  - USER_INVOCATION_ID
  - _SYSTEMD_USER_UNIT

Shall we:

Have a small PR now that just parses out severity
Make a PR to spec that proposes semantic conventions
Circle back to this after 2 is accepted

The plan looks good to me. And I think 1 should be just an additive change - we can keep the payload and do not remove PRIORITY for now.

One design option that comes to mind is something like the preset option used for severity parsing. Basically, we can have some predefined sets of values that will be parsed. One of these sets is the default. The user can optionally choose another set and/or add additional fields to any set.

This sounds good. Maybe even just fields would be enough. But would it mean that we have to come up with attribute names (semantic conventions) for all existing journald fields?

Maybe even just fields would be enough.

True. If you leave it unspecified you get all that we have semantic conventions for, otherwise you get what you specify.

But would it mean that we have to come up with attribute names (semantic conventions) for all existing journald fields?

I suppose we could make fields a map, where a the key is the journald field name, and the value is the name of the attribute. User can leave value empty if we have a default mapping for the field, otherwise must specify.

- type: journald_input
  fields:
    _BOOT_ID:            # has semconv => ok, defaults to semconv (e.g. "journald.boot.id")
    _COMM: "custom.comm" # no semconv, but user named it => ok
    _PID: "custom.pid"   # has semconv, but user overrode it => ok
    __CURSOR:            # no semconv => unmarshal error

I like this approach. I believe that not all the fields are going to be log attributes, some of them should be resource attributes, right? In that case this configuration model needs to be adjusted.

Yeah, makes sense. We can finalize design during implementation, but something to this effect should work well.

I'm fine with waiting for semantic conventions to be added to the spec.

I've pushed a suggestion for what this could look like. Feedback is welcome.

Some of the fields will be resource attributes, so I've added a fields and resource_fields config item with a default value and users can override.

For the moment, anything that does not have a semantic convention defined gets passed through unmodified. I've added a commented out list of all the fields defined in the documentation.

Do we also want to specify a list of fields that we drop by default? There may be unknown fields, which I guess should be kept if they are unknown (passed through unmodified), so we should probably only drop known fields if they don't get assigned a semantic convention.